The production of two-dimensional images that can be displayed to provide a three-dimensional illusion has been a long-standing goal in the visual arts field. Methods and apparatus for producing such three-dimensional illusions have to some extent paralleled the increased understanding of the physiology of human depth perception, as well as, developments in image manipulation through analog/digital signal processing and computer imaging software.
Binocular (i.e., stereo) vision requires two eyes that look in the same direction, with overlapping visual fields. Each eye views a scene from a slightly different angle and focuses it onto the retina, a concave surface at the back of the eye lined with nerve cells, or neurons. The two-dimensional retinal images from each eye are transmitted along the optic nerves to the brain's visual cortex, where they are combined, in a process known as stereopsis, to form a perceived three-dimensional model of the scene.
Perception of three-dimensional space depends on various kinds of information in the scene being viewed including monocular cues and binocular cues, for example. Monocular cues include elements such as relative size, linear perspective, interposition, light, and shadow. Binocular cues include retinal disparity, accommodation, convergence, and learned cues (e.g., familiarity with the subject matter). While all these factors may contribute to creating a perception of three-dimensional space in a scene, retinal disparity may provide one of the most important sources of information for creating the three-dimensional perception. Particularly, retinal disparity results in parallax information (i.e., an apparent change in the position, direction of motion, or other visual characteristics of an object caused by different observational positions) being supplied to the brain. Because each eye has a different observational position, each eye can provide a slightly different view of the same scene. The differences between the views represents parallax information that the brain can use to perceive three dimensional aspects of a scene.
Parallax information does not have to be presented to the brain simultaneously. For example, left and right eye depth information can be presented alternately to the left and right eyes, resulting in depth perception as long as the time interval does not exceed 100 msec. The brain can extract parallax information from a three-dimensional scene even when the eyes are alternately covered and uncovered for periods of up to 100 msec each. The brain can also accept and process parallax information presented to both eyes simultaneously if the parallax information is sequenced. For example. two or more views of the same scene taken from different observational viewpoints may be shown to both eyes in a sequence (e.g., each one of the views may be shown to both eyes for a short amount of time before showing the next view in the sequence).
Several three-dimensional image display methods have been proposed and/or implemented. These methods may be divided into two main categories of stereoscopic display methods and autostereoscopic display methods. Stereoscopic techniques including stereoscopes, polarization, anaglyphic, Pulfrich, and shuttering technologies require the viewer to wear a special viewing apparatus such as glasses, for example. Autostereoscopic techniques such as holography, lenticular screens, and parallax barriers produce images with a three-dimensional illusion without the use of special glasses, but these methods generally require the use of a special screen.
Other systems have been proposed, however, that require neither special glasses nor special viewing screens. These systems include autostereoscopic television and motion picture systems that utilize alternately displayed views of a scene recorded by two cameras from different points of view. For example, the devices described in U.S. Pat. No. 4,006,291 to Imsand; U.S. Pat. No. 4,303,316 to McElveen; U.S. Pat. No. 4,429,328 to Jones et al.; and U.S. Pat. No. 4,815,819 to Mayhew et al., all utilize two carefully aligned cameras to record horizontally, vertically, or a combination of horizontally and vertically displaced views of a scene. While these systems deal mainly with techniques of image acquisition for autostereoscopic display using standard screens, the cameras must be carefully matched and aligned to capture appropriate images. Further, once the images from the cameras have been captured, the alignment of the images cannot be readjusted.
In yet another approach, U.S. Pat. No. 5,510,831 issued to Mayhew describes a method of autostereoscopic display of parallax images using a slit scanning technique. In this technique, two cameras are carefully aligned to capture stereoscopic images. These images may be displayed by providing a first image as a background image and overlaying a second image onto the first image in the form of a scanning slit.
While each of these described methods and systems can be used to capture images for three-dimensional image display, there are problems associated with each. For example, many of the methods require the use of at least two carefully aligned cameras to capture images having parallax information. Aligning multiple cameras at a common scene is cumbersome. Not only are there multiple cameras to carry and to position, but proper alignment and color/luminance matching of the cameras can be difficult. Even after alignment, the cameras still may not provide a desired degree of image alignment for later display. Further, many of the prior art methods require special camera or lens mechanisms, video switching equipment, special viewing glasses, and/or special screens to create the three-dimensional illusion. Also, none of these three-dimensional display methods are suitable for use with randomly acquired images or with images extracted from a conventional video image stream (e.g., sequence) or images with parallel views, for example.
The present invention is directed to overcoming one or more of the problems associated with the prior art three-dimensional image display systems and methods.